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Higher education today is confronted with increasing marketization and aggressive regulation of 
the public sector. In an attempt to address these challenging issues, public universities are 
undergoing unprecedented change, particularly Colleges of Education. Redesigning educational 
leadership preparation programs, working in partnership with local school districts, and 
embedding field work are just some of the strategies that Colleges of Education have 
implemented to address these challenges. Now some states are requiring their public institutions 
to justify their educational leadership preparation programs ’ existence by showing the difference 
their program makes to their graduates and to the field. This article first discusses the reasons 
why educational leadership preparation programs need to show performance, value, and impact 
with a specific focus on Florida. Second, the article questions whether program faculty currently 
has the capacity to gather program impact data. Finally, the article describes strategies that 
program faculty can implemen t in order to develop robust impact statements. 
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The importance of clearly defining what successful learning or performance looks like 
has become increasingly evident during the past decade. Without a doubt, the better one 
understands what excellence looks like, the greater one’s chances are for achieving—or 
surpassing—that standard. (National Policy Board for Educational Administration, 2011) 

Program impact is an outcome measurement, “a systematic way to assess the extent to 
which a program has achieved its intended results. The main questions addressed are: 
What has changed in the lives of individuals, families, organizations or the community as 
a result of this program? Has this program made a difference? How are the lives of 
program participants better as a result of the program?” (The Evaluation Forum, 2000, p. 
9) 


Introduction 

Higher education has become “market-obsessed,” competing for students in a world where 
university marketing predominates, and where considerable effort is focused on ensuring that the 
product is perceived as superior to any other in the field. In 2003 David Kirp suggested that 
higher education as a market commodity is measured by student input (i.e., what students bring 
into the program) such as GRE scores, years of experience in the field, and job titles that imply 
success in the field. This process replicates the way the private market rewards firms with profits 
when they produce and sell units of the required quality. But this quantitative data tells a future 
purchaser of the program little about quality or consumer experience. In order to help the student 
consumer and to make the institutional choice less opaque, there are calls for increased 
accountability and improved assessment reverberating throughout all levels of higher education 
(Lydell, 2008; Pounder, 2011). 

Historically, the assessment of student outcomes for graduate education has been limited 
(Lydell, 2008). But in the last decade graduate education has begun to operate in an international 
context and it is therefore important that universities benchmark their programs in a global 
environment. Recently, educational leadership faculty have been forced to examine “what goes 
on” within their graduate programs and, more specifically, what outcomes for students this 
education yields (Lydell, 2008) in an attempt to both ensure and market program effectiveness 
and quality. In fact, the most recent newsletter from Division A, American Education Research 
Association (AERA, Spring, 2013), Learning and Teaching in Educational Leadership Special 
interest Group highlighted the importance of program evaluation in the international arena by 
focusing on “Linking Administrator Preparation Programs to Student Outcomes,” asking 
whether this is a “quixotic quest.” 

The focus of this paper, however, is program value and impact, not program evaluation. 
Program impact is increasingly important in the growing marketplace of higher education, 
particularly for public institutions delivering programs that address standards and criteria 
required by their state. In Article IX, Section 7(a), the Florida Constitution, establishes a system 
of governance for the State University System of Florida (11 public universities) “in order to 
achieve excellence through teaching students, advancing research and providing public service 
for the benefit of Florida’s citizens, their communities and economies.” One of the 
responsibilities of the Board of Governors is “avoiding wasteful duplication of facilities or 



programs.” To this end the State University System of Florida Strategic Plan 2012-2025, created 
by the Board of Governors, emphasizes three critical areas: Excellence, Productivity, and 
Strategic Priorities for a Knowledge Economy. Under the category of teaching and learning the 
governors highlight (a) strengthening the quality and reputation of academic programs and 
universities; (b) increasing degree productivity and program efficiency; and (c) increasing the 
number of degrees awarded in STEM and other areas of strategic emphasis. The governors state 
that “some unproductive academic programs are being re-tooled or terminated” (p. 7). The term 
“unproductive,” is concerning depending on the lens being used (e.g., efficiency, contribution to 
the workforce, contribution to research and development, or contribution to a specific discipline 
or field). 

Also in Florida, Senate Bill 1664 passed by the House (YEAS 110 NAYS 7) was 
presented to the Governor for signing, June 3 rd , 2013. An issue of concern for educational 
leadership faculty is the fact that the bill states that a person with experience in “executive 
management,” and a pass on the Florida Educational Leadership Exam (FELE) is eligible to 
obtain a temporary principal license with the proviso that they are mentored for three years. The 
proposer of the bill (Senator John Legg) believes this is in line with Florida’s commitment to 
education choice. This further emphasizes the increasing need for university preparation 
programs to be able to show their impact in the field. 

If all programs in public universities in Florida are being evaluated in the same manner, 
and if the role of the community/state colleges continues to move into areas that were previously 
the domain of universities (i.e., four year undergraduate degrees, teacher preparation), it is 
imperative that a university be able to show how a specific program can be differentiated from 
that in another public institution. This can be achieved by clearly articulating the value and 
impact of the program on the graduating individual, the educational field, school districts served, 
schools, and student achievement. If a university’s program in Florida is unable to show impact, 
they may be viewed as “unproductive,” making the program vulnerable to closure. 

Faculty in educational leadership programs need to develop the capacity to evaluate the 
impact of their preparation programs and, as a field, develop a more sophisticated understanding 
of the preparation approaches that contribute to the school improvement work of graduates who 
become educational leaders (UCEA website). A search of marketing materials for educational 
leadership preparation programs finds frequent use of the tenn “impact,” with a usual reference 
to program graduates making an impact in the field. Far too frequently, self-reported perceptions 
of administrators and teachers, accounts of program graduates’ employment patterns, and in 
some cases reported measurements of student outcomes in schools led by program graduates are 
used to illustrate impact. However, objective causal linkages between program features and 
particular leadership behaviors and their effects on organizational dynamics, teacher practices, 
and targeted student outcomes remain opaque (Orr, 2011). Data on time-to-degree indicators 
while a common measure evaluative measure reveal nothing about program impact or quality but 
a lot about program efficiency. 

Why Do Programs Need to Show Performance, Value, and Impact? 

Impact requires a program to change attitudes or behavior, or benefit society (Diem, 1997). 
Identifying program impact is important for four reasons: (a) to justify the investment of public 
funding; (b) to earn and build professional, organizational, and political credibility and support; 
(c) to yield tangible data (quantitative and qualitative) that enable a public institution to show the 



quality of their program(s); and (d) to satisfy the requirements of political bodies and funding 
agencies. Measuring impact is part of both the summative evaluation component, which judges 
the quality and worth of a program, and the formative evaluation component, which provides 
feedback to help improve and modify a program (University of Central Florida, Program 
Assessment Handbook, 2008). 

Do Educational Leadership Preparation Programs Have the Capacity to Show 
Program Performance, Value, and Impact? 

Currently, administrative preparation programs are under tremendous political pressure to 
demonstrate their value and impact on the perfonnance of leaders whom they prepare and 
ultimately the schools that they lead. There are over 500 university-based educational leadership 
preparation programs in the United States, all of which are accountable to state and national 
leadership preparation standards and graduates’ performance on the state leadership test (UCEA 
website, 2008). These university programs are now under scrutiny by critics external to the 
educational leadership professoriate who argue that educational leadership program content, 
rigor, and relevancy are generally suspect (Fordham Foundation, 2003; Hess, 2003; Hess & 
Kelly, 2005; Levine, 2005). Some states have responded to their critics—Iowa, Mississippi, 
Alabama, Tennessee, Florida, and Louisiana—by pressuring colleges and universities to update 
their educational leadership training programs. 

Arthur Levine, then-president of Teachers College, at Columbia University, concluded in 
2005 that many university-based school leadership programs are engaged in a “race to the 
bottom,” attempting to attract students by offering lower standards, ensuring less demanding 
course work, and awarding degrees in less time and with fewer requirements. Levine also noted 
that many programs fail to actualize the placement of school leadership graduates in 
administrative posts. In a parallel critique, the Southern Regional Education Board (SREB) 
report Schools Can 7 Wait: Accelerating the Redesign of University Principal Preparation 
Programs argues: 

There is a lack of urgency for refocusing the design, content, process, and outcomes of 
principal preparation programs based on the needs of schools and student achievement 
and little will happen until there are committed leaders of change at every level—state, 
university, and local school district. (Fry, Bottoms, & O’Neill, 2006, p. 4). 

A 2007 Wallace Foundation survey of 22 higher education institutions concluded that 
“many universities are not getting the job done . . . [they] have moved at a glacial pace to make 
improvements, or have made only cosmetic changes” (p. 10). Furthermore, the report criticized 
leadership faculties for being overly concerned with maintenance of existing course work (often 
posited as evidence of meeting required standards), faculty independence in course content 
development, and potential losses in enrollment that might translate to decreases in revenue 
production. 


Contextual Background: Development of Florida State Standards 

In 1979 the Florida legislature passed the Management Training Act (FS 231.086), which 
outlined a new system for administrator certification. This system involved three major 



partners—the state’s universities, the 67 school districts, and the Florida Department of 
Education (FDOE). Each partner was assigned roles or functions. CoEs were tasked with 
teaching the knowledge base associated with the field of educational leadership. The act also 
specified that principals should be trained in competencies, identified by the commissioner of 
education, necessary to execute school-based management. It also authorized school boards to 
submit to the commissioner a proposed program designed to train school leaders. 

Additional legislation in 1980 created the Florida Council on Educational Management 
(FCEM) (FS 231.087) to be controlled by the deputy commissioner of education, not CoEs. The 
council consisted of 17 appointed members (six principals, three university professors, three 
persons from private-sector management, one elected school superintendent, one appointed 
superintendent, one school board member, one person engaged in school district management 
training, and one person from the Department of Education). Duties of the FCEM included 
identification of requisite principal competencies, validation of these competencies, development 
of competency measurement and evaluation, production of policies for compensation, 
identification of screening, selection, and appointment criteria, and other related activities. The 
FCEM was also asked to create and adopt guidelines and a review process and procedures for 
program approval. CoEs’ educational leadership programs were directed to offer in their 
program the following eight core curriculum areas: (a) public school curriculum and instruction, 
(b) organizational management and development, (c) human resource management and 
development, (d) leadership skills, (e) communication skills, (f) technology, (g) educational law, 
and (8) educational finance, plus six credit hours in either elementary, middle, secondary, or 
exceptional student education. School districts, through their respective human resource and 
management development (HRMD) programs, were expected to provide “hands-on” 
(perfonnance of the 19 principal competencies adopted by the FCEM and all duties required by 
the district school board) through a type of administrative internship and 
management/principalship training program. The FDOE was given the responsibility for 
approval of Colleges of Education’s curricula, districts’ HRMD programs, and the administration 
of the required Florida Educational Leadership Examination (FELE). 

In 1981, FS 231.087 was retitled as the Management Training Act and included revisions 
and additions (such as recognition of the Florida Academy for School Leaders and the 
connection of selection, appraisal, and training programs to certification). 

In 1985, the legislature passed the Principals and Assistant Principals; Selection (FS 
231.0861). This act was important as it prescribed a deadline (July 1, 1986) for compliance of 
districts to adopt and implement their approved, objective process for the screening, selection, 
and appointment of principals and assistant principals. It also provided strict guidelines 
concerning out-of state administrators moving into Florida and examination requirements for 
new administrators. Also in 1985, the legislature renamed FS 231.087 Management Training 
Act; Florida Council on Educational Management; Florida Academy for School Leaders; Center 
for Interdisciplinary Advanced Graduate Study. This revision created district program approval 
guidelines and a training network, and emphasized 19 Principal Competencies as well as yearly 
accountability. 

Several state board rules were written and enacted in 1986 to provide criteria as written in 
the statutes for certification of principals and assistant principals. Florida School Principal 
Certification (SBER 6A-4.0081) addressed the requirements Levels 1, 2, and 3 Certification. 
Florida Educational Leadership Examination (SBER 6A-4.00821) concerned the written 
examination for certification (i.e., the Florida Educational Leadership Examination or FELE). 



Specialization Requirements for Certification in Educational Leadership—Administrative Class 
(SBER 6A-4.0082) required three years of successful teaching experience, a master’s degree, 
and successful completion of the Florida Educational Leadership Core Curriculum for initial 
certification. School Principal—Administrative Class (SBER 6A-4.0083) required 
documentation of successful performance of principalship duties through the Preparing New 
Principals Program. It also addressed the requirement of a district-based objective screening and 
selection system. 

With the sunset of the law in 1999 requiring HRMD programs based on the 19 
competencies, the state was required to develop new leadership standards. A Commissioner’s 
Educational Leadership Summit was held in September 2002 to bring together business, higher 
education, and school district leaders to discuss educational leadership. The summit outcomes 
were an agreement to develop new educational leadership standards modeled after the Educator 
Accomplished Practices (EAPs), creation of “Standards Working Group” to research and draft 
standards, and a peer review process to promote stakeholder involvement (Florida Department of 
Education website). 

In 2003 a series of meetings was held with representatives of school principal and higher 
education groups, along with selected school administrators, resulting in revisions to the Florida 
Principal Competencies and sample key indicators. Over 200 participants from 40 counties 
participated in the development of the standards. In addition, all current school principals in the 
state were sent the recommended standards and were asked to comment on them, along with 
representatives of the Florida Association of School Administrators (FASA), the Florida 
Association of District School Superintendents (FADSS), the Florida Association of Professors 
of Educational Leadership (FAPEL), and the Florida Department of Education (FDOE) (Florida 
Department of Education website, 2013). 

In 2005 the Florida Principal Competencies were replaced by the Florida Principal 
Leadership Standards (FLPS), State Board of Education (SBE) 6B-5.0012. They served as the 
state’s standards that Florida school leaders must demonstrate in preparation programs and in 
school administrator evaluations. FPLS were adopted into rule (6 A-5.080) by the State Board in 
2006-07, and Educational Leadership and School Principal Certification programs were 
redesigned to implement the new standards in 2008. 

In 2006 the Florida legislature passed the William Cecil Golden Professional 
Development Program for School Leaders (F.S.1012.986), which replaced the school districts’ 
HRMD plan that was designed to meet the “sunsetted” Management Training Act. This act 
included the following goals: (1) provide resources to support and enhance the principal’s role as 
the instructional leader; and (2) build the capacity to increase the quality of programs for 
preservice education for aspiring principals and in-service professional development for 
principals and principal leadership teams. 

In 2012 the legislature adopted the Fourth Edition of the Competencies and Skills 
Required for Certification in Educational Leadership in Florida. The revised competencies and 
skills reflect an alignment to the revised Florida Principal Leadership Standards, Rule 6A-5.080, 
Florida Administrative Code (FDOE memorandum, 2013). The FPLS currently form the basis 
for all of Florida’s leadership preparation programs and establish the core practices for 
leadership appraisal systems. Revisions to the competencies and skills consequently necessitated 
content changes across all three current FELE subtests resulting in a new examination, FELE 
3.0. 



Criticism of Colleges of Education 


Under the Student Right to Know and Campus Security Act (1990), higher education institutions 
are required to produce data on the following: (a) retention and graduation rates; (b) financial 
assistance available to students and requirements and restrictions imposed on Title IV aid; (c) 
crime statistics on campus; (d) athletic program participation rates and financial support; and (e) 
other institutional information, including the cost of attendance, accreditation and academic 
program data, facilities and services available to disabled students, and withdrawal and refund 
policies, to calculate and disclose a precisely and uniformly defined graduation rate. From the 
beginning there were doubts about the usefulness of using this rate to compare the effectiveness 
of institutions. The media were quick to create league tables, and, not surprisingly, opinion of the 
league tables among institutions varies depending on their ranking positions. 

Major stakeholders such as state and federal legislators perceive the US educational 
system as falling behind international competitors. The Spellings Commission report by the US 
Department of Education (2006) accused American higher education of becoming what “in the 
business world, would be called a mature enterprise: increasingly risk-averse, at times self- 
satisfied, and unduly expensive” (p. xii). The commission called on higher education institutions 
to develop “new performance benchmarks designed to measure and improve productivity and 
efficiency” (U.S. Department of Education 2006, pp. 14, 19, 20). The National Commission on 
Accountability in Higher Education issued a report in 2005 that made similar recommendations. 

But neither the Spellings Commission report nor the National Commission on 
Accountability in Higher Education explained how transparency, measurement, and 
accountability measures improve graduate education, specifically in educational leadership 
programs. 

As criticisms mount, there is an increasing sense of urgency among Colleges of 
Education to be seen as relevant and essential to school leadership preparation. Colleges of 
Education are cognizant of the current critical narrative that accuses them of falling short due to 
academia isolation, elitism, and lack of understanding of current problems occurring in the field. 
This narrative is constantly reiterated as state- and national-level policy actors, urban districts, 
foundations, and private organizations question how best to prepare leaders, particularly given 
existing shortages of highly qualified principals and superintendents and the complex demands 
of leading school reform efforts. 

Can Program Faculty Gather the “Best” Data? 

How do we know that a program is a “quality program” and that its graduates are “quality 
educational leaders?” The first indicator in Florida would be whether the program is state 
accredited. In Florida there are eight private institutions offering Educational Leadership (EL) 
Preparation programs, 12 public institutions, and one district with a total of 5,132 candidates and 
1,700 completers in 2011, the last year for which data is available. In 2012 the Florida 
Department of Education contracted with MGT of America Inc. (MGT) to review and examine 
the implementation of the Continued Program Approval Standards for teacher and school leader 
preparation programs in Florida’s 48 institutions of higher education, as required under Part D of 
the Great Teachers and Leaders’ portion of Florida’s Race to the Top (RTTT) application (MGT 
report, 2012). 



Further, the FDOE is also working with a Race to the Top (RTTT) committee, the 
Teacher and Leader Preparation Implementation Committee (TLPIC), to review the Continued 
Program Approval Standards, identify and recommend perfonnance measures, review 
curriculum, and recommend modifications to the site visit processes. The TLPIC is currently 
exploring new ways to assess and improve educational leadership preparation at the state level. 
Its work is cognizant of the work of the University Council for Educational Administration 
(UCEA), Educational Administration—Special Interest Group Taskforce on Educational 
Leadership Preparation, and the recommendation by the National Conference of State 
Legislatures (2012) who identified the need to redesign preparation programs, develop tougher 
program accreditation, strengthen licensure and certification requirements, and provide 
meaningful mentoring programs and quality ongoing professional development. Turning their 
attention to program input, they recommended higher standards in recruiting, selecting, and 
retaining a talented pool of aspiring principals, and finally they recommended evaluating 
candidate and program effectiveness. The need to find meaningful outcome measures is one that 
states, organizations, and institutions are wrestling with. 

Paradigm Shift: From Evaluation to Performance, Value and Impact 

Literature on educational administration program value and impact is limited. In 2011 the 
Context for Success project, sponsored by the Bill and Melinda Gates Foundation, brought 
together a group of scholars and policy makers to consider issues related to effective program 
evaluation focused on designing “input-adjusted metrics.” With the support of the Bill and 
Melinda Gates Foundation, the consulting firm HCM Strategists invited a number of scholars 
from around the country to write papers that would discuss the methodological issues in 
accounting for differences in student populations when evaluating institutional performance. In 
some cases these authors were also asked to demonstrate the effects of such adjustments using 
actual data (Clotfelter, 2012). Seven papers were commissioned, along with an overview and a 
literature review. These will be available in September on the project website at 
http://www.hcmstrategists.com/contextforsuccess/. 

The most authoritative study to illuminate the chain of causal influences between 
program design features and school outcomes was conducted by Orr and Orphanos (2011). 
Through the application of structural equation modeling, they found that the effect sizes of 
principals who graduated from exemplary preparation programs were significantly larger than 
principals who graduated from traditional programs on effective leadership practices, school 
improvement practices, and effective school culture. 

In 2012 Davis and Darling-Hammond examined five exemplary preparation programs 
and found that only one program had multiple and robust measures that linked program features 
with graduates’ effects on important school outcomes. The University of Illinois, Chicago, had 
developed an evaluation protocol that used a longitudinal approach with multiple measures of 
student and principal performance measures and comparative descriptive analyses between 
treatment and nontreatment programs, graduates (principals), and K-12 students in the Chicago 
public school system. 

The nonprofit urban principal preparation program New Leaders has also accumulated a 
considerable amount of evidence about their graduates who become principals, their leadership 
practices, and various school and student outcomes. Likewise, the New Teacher Center at the 
University of California, Santa Cruz, has developed an administrator preparation program 



evaluation protocol that aligns principal behaviors, attitudes, and retention data with student 
achievement outcomes. In addition, the National Institute for School Leadership is engaged in 
the development of strategies to assess the relationship between certain administrative practices 
and student achievement (Davis, 2013). 

The Council of the Great City Colleges of Education sponsors the Dr. Shirley S. 
Schwartz Urban Education Impact Award to honor an outstanding partnership between a 
university and an urban school district that has had a positive, substantial impact on student 
learning. Criteria for the award include employment data, leadership roles, and school value- 
added scores. In 2012 the College of Education at the University of Illinois, Chicago, won the 
award. It is worth noting that the award’s criteria included the school value-added scores. This 
remains a contentious criterion as program graduates not in an administrative position may be 
required to use their teacher scores, which can be used to compare program completers while 
they are in instructional or nonadministrative positions as well as when they receive 
administrative appointments. But is this an impact measure for educational leadership program 
graduates? 


Program Performance, Value, and Impact Strategies 

Unlike undergraduate education there is no graduate version of the Collegiate Learning 
Assessment that measures a program’s value added. In Florida, in order to graduate from a state 
approved program, a Beginning Effective School Administrator Candidate (BESAC) must pass 
the Florida Educational Leadership Exam (FELE). A standardized test of content knowledge that 
creates a standard measure of BESACs across all Florida-approved leadership programs. FELE 
as a measure of program effectiveness is therefore not influenced by intervening variables in the 
school setting impacting an accurate measurement of preparation program effects (FAPEL, 
White Paper, 2013). 

The National Governors Association recommended that colleges and universities be 
funded based on the effectiveness of their graduates in the workplace, and to this end the state 
collects program completion data and employment data. But there are pragmatic issues with 
both. First, certain accountability expectations and assessments, such as completion time to 
program graduation, while infonnative, are insufficient to enable programs to strengthen 
capacity to develop quality leaders capable of leading continuous school improvement for all 
children and their schools (UCEA website, 2008). Second, employment data is affected by the 
intervening time gap between program graduation and successfully achieving a principal 
position. Intervening variables, such as Level 2 Certification programs, and professional 
development impact the program graduate. Further, across the state there is a lack of consistency 
among school districts of the assigned coding of school administrators’ titles. Principal, assistant 
principal, dean, curriculum coordinator, community education coordinator, and any of the above 
as “interim” are counted as a school-based leadership position, but some may be instructional, 
nonadministrative positions (dean, curriculum coordinator, etc.). Personnel classifications are 
instructional/nonadministrativc or administrative or classified. BESAC students fall in to the 
instructional/nonadmini strati vc category and may remain there as teacher leaders either in 
schools or at the school district levels. Educational leadership program graduates require time to 
achieve a school leadership position. Further, time is then required to measure individual student 
and program impact in the field, assuming that appropriate methods can be found that are stable 
over a number of years. 



In trying to identify appropriate program impact rather than program evaluation 
strategies, it is useful to be cognizant of Bennett’s hierarchy of program cause and effect in 
response to a need to show program impact. Bennett (1975) developed a seven-step hierarchy 
that shows the causal links between the steps from inputs to outcomes. Program impact increases 
as you move through the hierarchy. 

1. Inputs: Costs, time, and personnel 

2. Activities: Workshops, field days, seminars, awareness-raising campaigns 

3. Participants: Number of people, their characteristics 

4. Reactions of participants: Degree of interest, likes and dislikes 

5. Changes in knowledge, attitudes, skills, and aspirations 

6. Practice change: Adoption and application of knowledge and skills 

7. End results: Range of outcomes that might be desired in program delivery 

Bennett pointed out that program designers must be able to answer the question: “How do you 
know this program was responsible for these impacts?” Bennett stated that the data required to 
measure program impact on clients (e.g., program graduates, school districts) can be gathered in 
only three ways: ask them, test them, or observe them. 

Recent studies reveal that researchers have used a variety of quantitative (e.g., descriptive 
and inferential statistics, regression analyses, structural equation modeling, hierarchical linear 
modeling) and qualitative methods (e.g., case studies, grounded theory, and ethnography) to link 
program features with graduates’ impact on school and student outcomes. However, much of the 
literature focuses on the aggregate characteristics of programs and their relationship to particular 
leadership practices, characteristics, or career outcomes (Darling-Hammond, LaPoint, Meyerson, 
& Orr, 2007; Hess & Kelly, 2005), which Bennett would categorize as low-level impact. Other 
research has assessed the relationship between preparation program elements and graduates’ 
career advancement and retention (Orr, Silverberg, & LeTendre, 2006; Orr, 2011), which would 
be categorized at a higher level in Bennett’s hierarchy. Current efforts to measure how programs 
impact particular principal practices and school outcomes could be categories in Bennett’s 
highest level of program impact. 

What Can Educational Leadership Programs Do to Develop Robust Impact Statements? 

First, program faculty should be clear why they are gathering and publishing program impact, 
performance, and visibility data. Who is the desired audience? The answer to this question will 
detennine how the impact data and statements will be used. 

An effective, impactful program must be informed by horizontal and vertical 
communication. By this we mean that program design and desired outcomes should be grounded 
not only on state standards but on the input of all stakeholders, specifically program graduates 
and school districts served by the program. Such collaboration aids the gathering of program 
graduate performance data at prespecified points after graduation both from the graduate and 
from an independent agency. The development of a collaborative partnership in program 
delivery places a moral imperative on partners and stakeholders to help in the gathering of 
program data, which always proves difficult whether instigated by the program, the university, or 
the state. 



Clifford, Behrstock-Sherratt, and Fetters (2012) described several program outcomes 
from which impact statements can be grounded, all of which reflect principal practice in the 
field: (a) a shared vision, (b) improving instruction, (c) a supportive learning environment, (d) 
managing resources, (e) positive relationships, and (f) ethical leadership. 

The development of impact statements currently tends to be dependent on data gathered 
by university alumni offices that for some time have collected data on program graduates’ 
employment status, promotion efforts, salary attained, and other additional data as requested by 
educational leadership faculty. However, educational leadership programs intent on gathering 
impact data need more specific information from their program graduates. Gathering such 
information is delicate and requires considerable open communication with the university’s 
alumni department regarding their own timeline and infonnation gathered. 

Conclusion 

The proliferation of a higher education market economy, alternative institutions, and alternative 
course delivery at traditional institutions require not only a reexamination of educational 
leadership graduate program assessment but also authentic awareness of program impact on the 
graduate and the school districts served. At the moment there is no known method that allows 
programs to establish irrefutable causal relationships between various program features and 
specific school outcomes. At best, analytic evaluation methods provide only estimates and 
approximations. This paper has argued that differentiating program evaluation from program 
impact is challenging but necessary for public universities offering an educational leadership 
preparation program. The development of reliable and accurate metrics for assessing the impact 
of educational leadership program graduates is promising and in the future will be the focus of 
considerable attention as it becomes paramount to the survival of some public university 
programs. 
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